Papers with soft labeling and distillation losses
SKD-NER: Continual Named Entity Recognition via Span-based Knowledge Distillation with Reinforcement Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Continual learning for named entity recognition (CL-NER) aims to enable models to continuously learn new entity types while retaining the ability to recognize previously learned ones. |
| Approach: | They propose a model that leverages knowledge distillation to retain memory and employs reinforcement learning strategies to optimize the soft labeling and distillation losses generated by the teacher model to effectively prevent catastrophic forgetting. |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets showing that it significantly improves the performance of the CL-NER task. |